This invention relates to small digital informational devices, such as personal digital assistants (PDA) and cellular mobile telephones, and, in particular to means for inputting graphical and spatial information into such devices.
personal digital assistants (PDAs) are pocket-size computers, provided with various software packages to carry out a variety of tasks, such as managing calendars and personal data bases, financial data management and analysis, communications and game playing. The user usually communicates with the device by means of a so-called touch screen, which is spread over the display screen, to form a combined interactive screen. Beside data and graphics to be read by the user, there are displayed on the screen also various graphic user interface (GUI) devices, such as menus, dialog boxes and lists, with which the user interacts by touching the screen at the appropriate pointsxe2x80x94with the tip of his finger, with a pen or preferably with a passive stylus. In a like manner, there may be displayed on the screen an image of a keyboard (i.e. a virtual keyboard); when any key image is touched by the user, a corresponding code is generated, similar to that generated by a conventional keyboard. With appropriate software, the stylus may also be used to draw lines and to write symbols, such as cursive alphanumeric characters, by moving its tip over the screen.
This method of graphic input into a PDA has two major drawbacks: (a) A touch screen is a relatively expensive component (compared to a normal display screen). (b) The screen is necessarily small, thus requiring the user to limit the range of movement of the stylus and to control the movement very finelyxe2x80x94which may prove undesirable for at least part of the user community. For virtual keyboard operation, the small size may prove particularly inconvenient, especially for multi-finger operation. In PDA devices that include a hardware keyboard, which generally is limited to twelve numeric keys (and is then called Keypad), such as cellular telephones, the above drawbacks are valid as well, whereby a keypad is even more limited in convenience.
For game playing and similar programs that involve spatial control of virtual objects on the screen, the usual manner of user interaction is by means of various control symbols and images on the screen, which the user touches with the same stylus. Such means of control are far from satisfactory, because they provide, at best, control along only two orthogonal axes at a time and because such control is not always analogous to the controlled entity of the virtual object; for example, a left-to-right motion of the stylus may control some angular orientation of the object or the rate of angular rotation. This, in many cases, proves to be inconvenient to the user and may lead to faulty operation.
There are various input devices known in the art and commonly used with conventional computers, such as a mouse, a so-called joy-stick and similar electromechanical devices, and various position input devices, also known as graphic tablets. Any of these may conceivably be connected to a PDA and thus used to effect the desired graphical input function and/or the desired control function, instead of the touch screen. All of them have, however, a major drawback in common, namely that their physical size is appreciable, compared with the size of a PDA, thus adding bulk to it and detracting from the usefulness of its pocket size. Also, each such device has additional drawbacks of its own. For example: A mouse does not provide any better control functions than as described above and is not handy for drawing lines and symbols; a graphic tablet is relatively expensive and is particularly bulky; and a joystick or its like serves best only for rate- and angular control but not for position control or for graphic input. Thus, to achieve most desired functions, at least two different ones of such devices need be deployedxe2x80x94which adds bulk and expense.
While a graphic tablet could conceivably be used for an external virtual keyboard, (thus avoiding the size limitations imposed by the display screen), no such applications have been known in the art (since so far no need has arisen). An additional drawback of the mouse and the graphic tablet, in common, is that they still provide only two-dimensional input at a time. Joy-stick like devices can provide control in more than two dimensions at a time, but their mode of operation is not analogous to the controlled variables (e.g. position, orientation and motion) of the virtual object, they usually control rate of motion (rather than position) and their movement is limited to a small volume around a fixed point. Various three-dimensional position input devices are known in the art, but all are even more expensive and bulky than the graphic tablet.
PDAs are already in ever widening use. Another type of a pocket-sized electronic device in widely increasing use is the cellular mobile telephone (CMT) with facilities for data communication and for personal data storage and management. There is a gradual increase in the number and sophistication of functions provided in CMT devices. Similarly to PDAs, CMTs also utilize a small screen to display data to the user. At present the prevailing means for input from the user is a keypad with twelve keys and several additional function buttons. It is expected, however, that, as the functionality of the CMTs widens, there will be a growing need for more graphics oriented and flexible input devices, similar to those needed for PDAs. It is even foreseen that the functions of a PDA and a CMT will eventually merge into a single device.
There is thus a need for, and it would be highly desirable to have, an input means for a PDA that will be compact and inexpensive and will enable convenient inputting of graphical and symbolic information. There is a further need for a compact and inexpensive means for inputting into a PDA three-dimensional graphics, and for controlling multi-dimensional position and orientation of virtual objects, displayed by a PDA, in a natural analogous-motion manner.
The present invention is applicable to PDAs, to function-enhanced CMTs, to any evolving combination of the two and to similar devices, including any that may evolve in the future. In the discussion to follow, as well as in the claims of the invention, all such devices will be referred to as PDAs, it being understood that the term PDA should be construed as applying to all and any of them.
Basically the invention provides a device, communicative with a PDA, that includes a hand-held movable part, or module, which inputs to the PDA information about all, or some of the six spatial variables of the movable part, namely its orientation about any of three orthogonal axes and the position of some reference point therein along any of three orthogonal axes. The movable part has preferably the form of a stylus, the size of an average pen, the reference point being near its tip, and will henceforth be referred to as a stylus. The device also includes a stationary part, or module, which is the one that communicates with the PDA; it may be an integral part of the PDA, mechanically attachable to it or a mechanically separate module. The device also includes a processing module, or processor, which receives electrical signals related to the spatial variables of the stylus and converts them into the desired coordinate- and orientation values, to be input to the PDA. As discussed in the Background section, the term PDA is used throughout this disclosure and application to represent any small digital device that provides display of data and graphics on a screen, such as a personal digital assistant or a cellular telephone with advanced data handling capabilities. The spatial variables, that isxe2x80x94position and/or orientation values in terms of two or three dimensions, fed into the PDA, are used to control various variables in programs running there. Examples of variables, which are frequently thus controlled, are analogous spatial coordinates of graphic elements and of simulated (virtual) objects. These and others are usually represented by graphic elements on the display screen of the PDA, so that the input spatial variables also control (albeit indirectly) position and orientation of graphics on the screen, thus forming an interactive loop with the operator of the device.
Any suitable technology may be utilized to embody the device, but the one described herein is preferred because of its good performance, the compactness that it enables and the low cost that it entails. This technology basically calls for low-frequency electromagnetic radiation, emitted from a plurality of sources disposed in the stationary module at some mutual distances and orientation, and one or more sensors for such radiation disposed in the stylus, one preferably being near the tip. The position and orientation values may be calculated from the strengths and relative phases of the signals resulting from sensing the radiation from the various sources. It is a feature of the technology, and hencexe2x80x94of the invention, that the distances between the sources may be small with respect to the region of space over which the stylus may be usefully moved. In the preferred embodiment, the radiation sources are electrical coils that generate magnetic fields and each sensor includes at least one device, in which each of the magnetic fields induces a corresponding signal. The spatial information is calculated in the processor, and supplied to the PDA, repeatedly. The processor also tracks any of the spatial variables, using estimation techniques.
In a first configuration of the invention, there is conveyed information about at least two spatial variables, namely position of the reference point along two axes. This information is useful in a wide variety of applications that may be programmed into the PDA, notably graphic applications, such as line drawing, object drawing and handwriting. To the extent that any program provides a drawing paradigm for more than two dimensions, information about corresponding variables of the stylus may be input and utilized therefor. Additionally the information may be used to move a cursor on the display screen and to effect various pointing, scrolling and selecting functions, as are commonly provided in graphical user interfacesxe2x80x94in the same manner as information from conventional input devices, such as a mouse, a touch pad and a joystick. Both absolute- and incremental modes of control may be exercised, the latterxe2x80x94with the help of a button-activated switch on the stylus.
In a second configuration of the invention, information about at least two position variables of the stylus tip, preferably all three position variables, is converted into codes corresponding to keys of a keyboard. In typical operation, there is provided some surface, external to the PDA (though it may conveniently be attached to it), on which is drawn the image of a keyboard. The user places the tip of the stylus within the marked image of any key, to effect generating the code of that key for inputting it to the PDA. In case of using only two-dimensional spatial information, the act of touching the surface is sensed by a suitable switch in the stylus. In the preferable case of three-dimensional spatial information, the act of touching may be detected by calculating the proximity of the tip to the surface and comparing it with some threshold. The three-dimensional mode also allows orienting the surface of the drawn keyboard at any angle with respect to the stationary module; a simple procedure, programmed into the processor and requiring the user to touch two or three marked points on the board, enables adapting the device to the board""s orientation. A keyboard of any size (within the range of the device), with any layout and any symbology, may be drawn and used in the manner described. For any such keyboard a one-time calibrationxe2x80x94or teaching procedure, programmed into the processor, is required.
In a modified form of the second configuration, there are a plurality of movable parts, which are preferably formed so as to be attachable to fingertips. These may operate, in conjunction with the single stationary part, to enable entering key positions on a keyboard by multiple fingersxe2x80x94akin to multi-finger typing.
In a third configuration of the invention, information about three-dimensional position of the tip of the stylus, as well as about orientation of the stylus about at least one axis is conveyed to the PDA in order to control, in a geometrically analogous manner, variables of any virtual object within a game program or a simulation program or the like. Both absolute- and incremental modes of control may be exercised, the latterxe2x80x94with the help of a button-activated switch on the stylus.
In another aspect of the invention, there is provided a method for typing, i.e. entering key symbol codes, into a PDA by touching key regions in a keyboard image on any surface external to the PDA, using the apparatus disclosed herein.
There is thus provided, according to the invention, an input device, communicative with a PDA, for inputting thereto the position of a movable reference point along at least two orthogonal axes, the PDA having a display screen, the device comprising:
a stationary module, communicative with the PDA and emitting electromagnetic radiation, and
a movable module, in relation to which the reference point is defined and which is receptive to the radiation;
wherein the positions that may be input are at substantial distance from the display screen and their range, along each of the axes, substantially exceeds any dimension of the stationary module.
According to further features of the invention the stationary module includes at least two sources that emit the radiation, each source emitting the radiation at frequencies of less than 100 KHz.;
the movable module includes at least one sensor, operative to receive the radiation and to generate electrical signals in response to the reception of the radiation; and
the device further comprises a processor, operative to process the electrical signals into position information.
According to other features of the invention, the processor is further operative to convert the position information into coordinates for positioning a cursor within the display on the screen of the PDA.
According to yet other features of the invention, there is defined a plane and within the plane there are defined key regions corresponding to keys of a keyboard and the processor is further operative to convert the position information into codes corresponding to the key regions. According to additional features, the processor is further operative to convert the position information also into a distance value along a direction normal to the marked external surface and to convert the distance value into a binary proximity indication, which signifies touching the surface.
In a common configuration of the invention the movable module is formed as a stylus; in an optional configuration there is a plurality of movable modules and they are formed as attachments to fingertips.
In an alternative configuration, according to the invention, there is provided an input device, cooperative with a PDA that includes a display screen, the device comprising a stationary module, communicative with the PDA, a movable module and a processor; whereinxe2x80x94
the movable module has a reference point and is operative to generate electrical signals that are related to the position of the reference point and to the orientation of the movable module, and
the processor is operative to process the electrical signals into at least four corresponding streams of values, to be fed into the PDA, each representing, respectively, any of the six variables associated with the movable module, namely the position of the reference point along three orthogonal axes and the orientation of the movable module about three orthogonal axes.
According to further features, the stationary module includes at least three sources that emit electromagnetic radiation, each source emitting the radiation at frequencies of less than 100 KHz.; and
the movable module includes at least one sensor, operative to receive the radiation and to generate electrical signals in response to the reception of the radiation.
In another alternative configuration, there is provided a PDA comprising a display screen and an input device, for inputting into the PDA the position of a movable reference point along at least three orthogonal axes, the input device including:
a stationary module;
a movable module, in relation to which the reference point is defined and which is operative to generate electrical signals that are related to the position of the reference point; and
a processor, operative to process the electrical signals into position information;
wherein the range of positions that may be input, along each of the axes, substantially exceeds any dimension of the stationary module.
In another aspect of the invention, there is provided a method for enabling a user of a PDA to key thereinto symbols of a keyboard by touching, or nearing, corresponding marked key regions on any surface external to the PDA, the method comprising:
(i) providing an input device that includes at least one movable part, each movable part having a reference point, the device being operative to generate is signals that represent the position of each of the reference points along at least two orthogonal axes;
(ii) for any key region within which any of the reference points is positioned, converting the resultant signals into a code signifying the corresponding key.